Fixing incorrect variable name 'size_gb' which actually represents 'size_mb'#4409
Fixing incorrect variable name 'size_gb' which actually represents 'size_mb'#4409
Conversation
There was a problem hiding this comment.
Pull request overview
This PR corrects a misleading fio API parameter name (size_gb) that actually represented a MiB value, and updates several test suites to pass sizes using the corrected size_mb keyword.
Changes:
- Renamed fio size parameter from
size_gbtosize_mband updated fio command construction accordingly. - Updated storage/power/cpu/performance test call sites to use
size_mb(and in some cases convert GB → MB explicitly).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| lisa/tools/fio.py | Renames fio size parameter to size_mb and uses it to build --size=<n>M. |
| lisa/microsoft/testsuites/storage/storagesuite.py | Updates fio call to use size_mb with a GB→MB conversion. |
| lisa/microsoft/testsuites/power/common.py | Updates fio call to use size_mb with a GB→MB conversion. |
| lisa/microsoft/testsuites/performance/storageperf.py | Changes NFS fio size to 256 * 1024 MB (substantial workload change). |
| lisa/microsoft/testsuites/performance/common.py | Fixes fio invocation to pass size_mb using the correct keyword. |
| lisa/microsoft/testsuites/cpu/cpusuite.py | Updates async fio call to use size_mb with a GB→MB conversion. |
Comments suppressed due to low confidence (1)
lisa/tools/fio.py:103
- Renaming the public keyword parameter from
size_gbtosize_mbis a breaking API change for any out-of-tree callers that still passsize_gb=(or rely on the previous positional argument order). Consider keeping backward compatibility by accepting both names (e.g., a deprecatedsize_gbalias that maps tosize_mb) for at least one release cycle, or raising a targeted, actionable error whensize_gbis provided.
def launch(
self,
name: str,
filename: str,
mode: str,
iodepth: int,
numjob: int,
time: int = 120,
ssh_timeout: int = 6400,
block_size: str = "4K",
size_mb: int = 0,
direct: bool = True,
gtod_reduce: bool = False,
group_reporting: bool = True,
overwrite: bool = False,
| time: int = 120, | ||
| ssh_timeout: int = 6400, | ||
| block_size: str = "4K", | ||
| size_gb: int = 0, | ||
| size_mb: int = 0, | ||
| direct: bool = True, | ||
| gtod_reduce: bool = False, |
There was a problem hiding this comment.
PR hygiene: the PR is marked as a bug fix but the description lists no related issue. Consider linking an issue (or explicitly stating why none exists) for traceability.
| time=0, | ||
| block_size="", | ||
| size_gb=100, | ||
| size_mb=100, # Keep fio data size at 100 MiB to bound test runtime. | ||
| group_reporting=False, | ||
| do_verify=True, |
There was a problem hiding this comment.
verify_disk_with_fio_verify_option runs fio in a for _ in range(100) loop with time=0 (no --runtime) and size_mb=100 * 1024, so each iteration will attempt to write+verify ~100 GiB and the loop can generate ~10 TiB of IO. This is likely to make the functional test impractically slow and/or hit CI timeouts. Consider keeping the previous effective size (100 MiB), or switching to time_based=True with an explicit runtime, or reducing the iteration count/size with a clear rationale. Also the inline comment says “100GB” but the expression is 100*1024 MiB (GiB).
| block_size="1M", | ||
| overwrite=True, | ||
| size_gb=1, | ||
| size_mb=1024, # 1GB |
There was a problem hiding this comment.
The new size_mb=1024 comment says “1GB”, but fio’s --size=<n>M uses MiB units. If the intent is 1 GiB this is fine; otherwise consider either using 1000 MiB for 1 GB (decimal) or updating the comment/variable naming to avoid unit confusion (GB vs GiB).
| size_mb=1024, # 1GB | |
| size_mb=1024, # 1 GiB |
| time=fio_run_time, | ||
| block_size="1M", | ||
| size_gb=fio_data_size_in_gb, | ||
| size_mb=fio_data_size_in_gb * 1024, |
There was a problem hiding this comment.
fio_data_size_in_gb is multiplied by 1024 to produce size_mb, which corresponds to GiB→MiB conversion. Consider adding a short inline comment or renaming the variable to avoid confusion between GB (decimal) vs GiB (binary) units, since fio’s --size=<n>M is MiB-based.
| size_mb=fio_data_size_in_gb * 1024, | |
| size_mb=fio_data_size_in_gb * 1024, # fio --size=<n>M uses MiB; convert GiB to MiB. |
| def launch( | ||
| self, | ||
| name: str, | ||
| filename: str, | ||
| mode: str, | ||
| iodepth: int, | ||
| numjob: int, | ||
| time: int = 120, | ||
| ssh_timeout: int = 6400, | ||
| block_size: str = "4K", | ||
| size_gb: int = 0, | ||
| size_mb: int = 0, | ||
| direct: bool = True, | ||
| gtod_reduce: bool = False, |
There was a problem hiding this comment.
Renaming the public Fio.launch(...)/launch_async(...)/_get_command(...) parameter from size_gb to size_mb is a breaking change for any out-of-tree callers using the old keyword. If backward compatibility is desired, consider temporarily supporting both names (e.g., keep size_gb as a deprecated alias that maps to size_mb with a warning) and remove it in a later major release.
…ize_mb' Fixing incorrect variable name 'size_gb' which actually represents 'size_mb' Co-Authored-By: Copilot <175728472+Copilot@users.noreply.github.com>
c3afa4e to
34396b8
Compare
| fio_data_size_in_mb = 1 | ||
| try: | ||
| image_folder_path = node.find_partition_with_freespace(fio_data_size_in_gb) | ||
| image_folder_path = node.find_partition_with_freespace(fio_data_size_in_mb) | ||
| # Each CPU takes ~10 seconds to toggle offline-online |
There was a problem hiding this comment.
node.find_partition_with_freespace takes size_in_gb, but this call now passes fio_data_size_in_mb. That makes the unit semantics incorrect (the value will be treated as GiB), and could cause unexpected failures or over-strict disk selection if fio_data_size_in_mb is adjusted. Consider keeping a separate required_space_gb variable (or converting MB→GB) when calling find_partition_with_freespace.
| fio_data_size_in_mb = 1 | ||
| try: | ||
| image_folder_path = node.find_partition_with_freespace(fio_data_size_in_gb) | ||
| image_folder_path = node.find_partition_with_freespace(fio_data_size_in_mb) | ||
| # Each CPU takes ~10 seconds to toggle offline-online | ||
| fio_run_time = 300 + (node.tools[Lscpu].get_thread_count() * 10) | ||
| fio_process = node.tools[Fio].launch_async( |
There was a problem hiding this comment.
The constants fio_data_size_in_mb = 1 and the base 300 seconds in fio_run_time = 300 + (...) are magic numbers that control test behavior but aren’t explained inline. Please add brief rationale/comments (or named constants) so future changes don’t accidentally skew workload size or runtime assumptions.
| def launch( | ||
| self, | ||
| name: str, | ||
| filename: str, | ||
| mode: str, | ||
| iodepth: int, | ||
| numjob: int, | ||
| time: int = 120, | ||
| ssh_timeout: int = 6400, | ||
| block_size: str = "4K", | ||
| size_gb: int = 0, | ||
| size_mb: int = 0, | ||
| direct: bool = True, |
There was a problem hiding this comment.
This PR is labeled as a bug fix, but the PR description lists no related issue. Consider linking the related issue (or a tracking work item) for traceability, especially since this changes a public API keyword argument on Fio.launch/launch_async.
| if size_gb: | ||
| cmd += f" --size={size_gb}M" | ||
| if size_mb: | ||
| cmd += f" --size={size_mb}M" |
There was a problem hiding this comment.
just change this into cmd += f" --size={size_gb}G" should be enough.
There was a problem hiding this comment.
If we do this all fio tests will run against GB ( * 1024 ) of current values. Which will result in timeouts as the time required for fio file size in GBs will be more than MBs.
For example some tests use size_gb=8192 which will be 8TB instead of current test file size 8GB.
One of the NFS testcase uses 100MB now in a loop of 100 iteration. With this suggestion it will become 100 GB * 100 = 10TB data reads and writes.
There was a problem hiding this comment.
"perf_disk()" accepts the value as 'size_mb' but passing the value to "fio.launch()" as 'size_gb'.
Please check: https://github.com/microsoft/lisa/blob/main/lisa/microsoft/testsuites/performance/common.py#L159
This is a typo issue not a test case issue.
There was a problem hiding this comment.
If you want to run current fio tests against 1023GB like we had in lisav2, I can raise a different PR for that after through testing. This is for just fixing the typo which impacts testcases not just disk perf tests but CPU and Power test cases as well.
| fio_data_size_in_mb = 1 | ||
| try: | ||
| image_folder_path = node.find_partition_with_freespace(fio_data_size_in_gb) | ||
| image_folder_path = node.find_partition_with_freespace(fio_data_size_in_mb) |
There was a problem hiding this comment.
the unit of this method find_partition_with_freespace is gb
| if size_gb: | ||
| cmd += f" --size={size_gb}M" | ||
| if size_mb: | ||
| cmd += f" --size={size_mb}M" |
There was a problem hiding this comment.
@SRIKKANTH thanks for fixing this, but I still would like to just change cmd += f" --size={size_gb}M" into cmd += f" --size={size_gb}G"
There was a problem hiding this comment.
Let me put the PR in draft until I finish the validation with you suggestion.
|
lili asked me to raise a PR for cmd += f" --size={size_gb}M" into cmd += f" --size={size_gb}G" . This requires more testing as this is going to impact multiple testcases. They all need to be tested and may need changes related to fio file sizes. Its needed to ensure that the fiofile size changes are not impacting the performance numbers and test execution times as well. This will be an elaborated exercise instead of simple type fix as I did in this PR. Hence closing this PR and I will raise the new PR when that is ready. |

Description
Fixing incorrect variable name 'size_gb' which actually represents 'size_mb'
Related Issue
size_gb is misleading variable name.
Type of Change
Checklist
Test Validation
Key Test Cases:
perf_nvme
perf_premium_datadisks_4k
perf_premium_datadisks_1024k
Impacted LISA Features:
None
Tested Azure Marketplace Images:
Test Results